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Overview 


An apparent phenomenon which defies the theory of probability occurs 


when Subject 2 plays this experimental game. He significantly exceeds his 


probability of success, .25, by scoring over .29. The question that this 


report addresses is: Is there a statistical or logical reason why he did 


so well? The methodology used to attack this problem and the resulting 


conclusions are summarized below. This summary can also serve as an outline 


to this detailed report. 


I. 


II. 


Statistical Analysis of the Machine Experimental Data 

Pre-experiment data analysis discovered a non-random characteristic 
through the examination of forward-backward state transitions (i.e., | 
Red-Blue, Blue-Red). However, the coefficient of correlation between 
the forward and backward states of .538 for the experimental data, .49 
for Machine 1 data and .48 for Machine 2 data were considered .low 
enough that this approach was dropped. Pre-experiment state transitions 
had a coefficient of sorretatton of .93. 

The experimental data randomness analysis consisted of axamining 
the distribution of color totals and the distribution of each color 
taken over various combinations and permutations of the data. No evidence 
of non-randomness was discovered. 

Analysis of the Subjects' Data Responses 


The subject's responses were analyzed with the emphasis on the 


discovery of his strategy or the unveiling of a trend which would give 


* him a statistical advantage. The possibilities investigated produces 


no solid reason how he was able to be so successful. However, in one: 


case there is a strong indication why he was able to succeed. It appears 


that. he was learning the states of Machine 2. The details of this are in 
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. the remainder of the report. | - 
ed Miscellaneous 
| The report contains a section entitled "Miscellaneous" for the 
purpose of displaying detailed data which wasn't directly required by 
the above more general analysis. Details such as how many successful 
choices in the color red during the 50th trial were there, or what was 
the relationship of the number of passes to the number of successes. 
The terminology used is as follows: the term "trial" refers. to the 
string of machine states and corresponding choices from the time the 
subject begins until. he makes 25.non-passing choices. A sample is a 
“machine state and/or subject choice (including passes). There are 


(25 + # passes/trial) samples in each trial. 
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I. . Statistical Analysis of the Machine Experimental Data 


Forward-backward State Transition Analysis SG1l 


In a previous memorandum (Memo ORD 2240-75, 12 June 1975 tof 
the question of randomness with the emphasis on state transitions as an indication 
of non-randomness was addressed. The data used in the investigation consisted 
of pre-experiment trials. The purpose of this section is to do a similar 
investigation using the actual data which occurred during S2's experiment. 

Table 1 presents all possible transition frequencies. All transitions 


‘should have equal probability. 


YELLOW ~ GREEN BLUE RED 
YELLOM | 204 199 199 216 
GREEN ee 192 218 222 207 
BLUE 217 206 228 222 
RED es 209 206 223 221 


Restructuring into a two-by-six table as in Ref 1 produces : 


Y/G Y/B- Y/R G/B G/R B/R 
FORWARD 199 199 216 222 . 207 222 
BACKWARD 192 211 209 - 206 206 223 


The conclusion based on pre-experimental data was that these state-pairs 
show a very strong relationship between forward and backward transition 
_ frequencies (coefficient of correlation =,93). However, computing the coefficient 
of correlation, Poo svtiidl data = .58, it becomes apparent that the 
degree of dependence is slightly Reduced: Therefore the dependence of 
forward to backward states can no longer be considered as a strong indicator 


of non-randomness. 
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The data used in the above discussion consisted of trials from both 
machine 1 and machine 2. Since non-randomness, made apparent by the state 
transitions, clearly existed fois presexveninentat data, the investigation of 
the experimental data continued to include a search for this trend in the 


individual machines. The transitions (including identity) are as follows: 


Machine 1 
YELLOW GREEN “BLUE. RED 

YELLOW 6 790 88 92 
GREEN 85 ts BF | 86 88 
BLUE 85 43 82 90 87 
RED 91 | 91 83 92. 
Machine 2 | . 

| YELLON GREEN ~ BLUE RED 
YELLOW 108 120 1 124 
GREEN. 107 131 136 119 
BLUE. 126 124 138 135 
RED 118 15 140 129 


Computing the two coefficients of correlation, 


= ,4934 


P machine 7] 
s2 data 
and 
P machine 2 7 -4838 
s2 data 


it is obvious that the forward and backward transitions are even less dependent 
than in the combined case. Thus ended the search for non-randomness througn 


state transition. 
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As a by-product the following table is produced for general information. 


BOTH MACHINES MACHINE 1 MACHINE 2 
. “MEAN “SD MEAN SD MEAN SD 
FORWARD 210.8 10.7 86.6 4.27 124 9.74 
BACKWARD | 207.8 ~ 9.00 86.2 oa 92 | 121 11.25 
TOTAL DATA POINTS 3483 | 1446 2037 
COEFF OF COV 5843 4934 - 4838 
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The machine data used during the S2 experiment has been combined, summarized 
and/or permuted in an attempt to establish evidence or randomness or non- 


randomness. If an obvious indication of non-randomness would have evolved 


this task would be simplified because it would have become a closed form 
problem (se the solution would be - the data has non-random characteristics). 
However, what has resulted is that various forms_of the data have been examined 
with all indicating that the data is random. | | 

Tables, plots and eonmentany are presented in this section to demonstrate 
randomness and in some cases just. to provide general information concerning 
the machines data. : 


The distribution of the colors collectively and for each machine is 


as follows: 

| Yellow Green Blue | Red Total Mean 
Machine 1 * 365 353 356 372 | 1446 361.5 
Machine 2 475 c 505 538 519 ~ 2037 509.25 
TOTAL 840 858 89) 89] 3483 870.75 


Machine 1 was not used in as many trials as machine 2 (44 trials to 56 
for machine 2), thus the difference in totals. The standard deviation of 
binomial distribution with n=3483 and p=1/4 is 25.56 which would imply that 
each separate number is reasonably close to. the mean. 

. Accepting the aieinibation of the totals eoneiaer the distribution of the 
colors throughout the experiment. The popluations used for this investigation 
consisted of the first 25 samples of each trial (100 trials total). This 
population is acceptable since the distribution of its totals was reasonable 
and since the performance of S2 was approximately the same (success-29.61%) 


for this subset. 
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_ The following three approaches comprise the strategy used to attack 
‘ane? the question of color distribution. 


1. Each trial (abbreviated to 25 samples) as analyzed separate interval. 


ee 


Obviously this will indicate any bias within each trial. 


(2. The data (2500 samples) is divided into intervals of five samples 


~ 


each, This will indicate unusual repetitions either within the 


interval or interval-by-interval. 


3. . The data is reformatted into 25 intervals of 100 samples, where the. 


nth interval consists of the nth sample an each trial. 
The resulescor approach 1 is shown in Figures 1.1.a, 1.1.b, 1.1.c, and 
1.1.4. 
The binomial distribution for this strategy (n=25 p=1/4) is mean 6.25 


| “and the variance 4.69. The plots indicate randomness throughout the 100 


trials. 
na | The results of approach 2 are similar to approach 1 and are shown in 
the four tables in Figure 1.2. The plots indicated randomness but are not 
shown because of monotomy. The binomial distribution mean is 1.25 and the 
variance o94:., | 
é The binomial distribution mean and variance for approach 3 is 25 and 18.75 | 
respectively (Figure 1.3). A plot of the data (Figure 1.4) for the "RED" 
case because of the concern for the higher variance and: ranges. ~The 13th sample 
seems to have an unusually high frequency of "RED" (44%). However in general | 
this investigation has not produced a significant non-random characteristic. 
aed 
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Figure 1.1.a Distribution of Machine Yellows Over Trials 
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Figure 1.1.b Distribution of Machine Greens Over Trials 
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Figure 1.3 Distribution of Machine Colors When Samples are Taken 100 at a Time 
. (One From Each Trial) 
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Figure 1.4 Distribution of Machine "Reds" when the Samples are taken 100 at 


a time (one from each trial) 
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Approach 1 has been repeated for Machine 1 and Machine 2 separately to 


check for abnormalities. The binomial distribution mean and variance are 


as follows: 
Trials Mean Variance 
Machine 1 44 11 8.25 


Machine 2 | 56 4 10.5 
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Best Strategy 


Based. on the above analysis what is the best strategy to pursue? No 


good strategy is available based on the randomness of the data. The best 


possible strategy based on the above transition matrices is: 
1. If the subject can't distinguish between machine Shek press blue 
- when blue appears, else pass. | 
2. If the subject can distinguish them on Machine 1, press yellow 
when yellow occurs, and on Machine 2 press. blue when red occurs. 
For all its worth, of the existing data the following success would 


result - 26%, 26%, and 27%. 
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Analysis of S2 Data Responses 
| The attempt here sb discover a reason for S2's success at responding. 
The investigation was unable to give a definitive reason for his success: 
Although no strategies waredincevered there was in ene case a indication that 
the subject was learning. . 

Two major approaches have been eakei in this investigation. They are. 


bas follows: j - | | rh by be” 


1. Strategy of $2 - Was there any trends in the way he guessed? Did, har! 
he respond based on the previous state of the machine? £ 


2. Hit analysis - Did the subjects’ hits (correct choices) increase 
within a run; did it increase from run to run (i.e., was he 
learning?) | 
Strategy of S2 
For general information and future reference the first figure (Figure 2.1) 
presented is the actual choices. One item of curiosity from this is that | 
. when he passes, he tends to doit. in strings. This characteristic of course 
wasn't pursued because of its insignificance to this report; however, observations 
like that are pointed out throughout the report as possible importance to those 
in the field. - 


Total Color Choices 


The distribution of $2's color choice totals are shown below. . 
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Figure 2.1 Subject 2 Color Choices for First Fifty Trials (0-yellow, 1-green, 
all | 2-blue, 3-red, 7-pass) 
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Figure 2.1 (Continued) $2 Color choices for Last 50 Trials 
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. Yellow Green Blue Red - 
Total Times Chosen - 881 All 237 977 
% of Total 35% 16.5% 9.5% 39% 


The first inclination is to try and determine how his strategy of choosing 


so many yellows and reds benefitted him. Examine the following table: 


. Yellow Green Blue — Red 

3 Qo 
Total Number of Hits 255 127 ké 292 
% of Total Hits 38% 17% 8% 40% 


% Of Success in Color 29% 31% . 25% 30% 
: (Hits - Correct Choices) 


As can be seen his results with blue are significantly lower than the others. 
However, assuming the probability of success to be 2” and using the pivemias 
distribution the expected value =69 and the standard deviation = 7. The 
inference from this is that the 60 Blue hits are not a statistical abnormality. 


However, it is curious that he did so much worse on his lowest preference. 


State Transition Color Choice 


This investigation consists of examining the states of the machine verses 
the choice on the next sample of the subject (i.e., if the machine shows "red" 
does the subject consistently choose one color on the next turn). Consider 


the following table: 


agus _ Machine , 

MACH’ Yellow Green Blue - Red Pass % Pass 
Yellow | 106” n9 «69 314.210 26% 
Green 177 25 69 316 252 30% 
Blue 2a] 99 27 198 302s 35% 

_ Red 322 157 65 oe 218 25% 

re 1 BO 
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The subject obviously avoids repeats (i.e, he assumes the machine 


~ won't repeat a color) which, based on the machine data analysis, isn't a 


strategy which would give him a statistical advantage. Previous analysis 


showed that identify transitions are approximately equally probable as 


_nonidentity. Notice also that he passes 35% of the time after seeing a blue. 


The same state transitions are shown below separated by machine. 


. Yellow 
Yellow | 48 
Green 62 
Biue | 105 
Red 133 
Yel low 58 
Green 5 
Blue | 136 
Red 189 


Green Blue 
49 25 
13 35 
36 10 
72 30 

weay fF 
70 44 
12 34 
63 17 
85 35 


Red 
150 
153 
78 
58 


164 
163 


120 


39 


Pass 
83 
83 


154 


The negative state transition (i.e., relationship of the subject color 


choice to the machine state on the next sample) is considered too bizarre of 


a concept to be presented in this section. Results of that investigation 


is found in the section entitled "miscellaneous" 
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Hit Analysis 

This section is significantly more important than the randomization 
analysis of the machine data. The reason is that if he is not learning from 
the machine or he is not taking advantage of biases then the discovery of 
such non-randomness is of little value to the overall analysis. 
Learning from Trial to Trial | 

The question of whether the subject learned from trial to trial can 
bask be answered by examining the following three plots.. The first is the 
number of hits vs. the trial number, the second is a frequency distribution 
Of the number of trials vs. number of hits, the third is the accumulated 


probability vs. the trial number. 
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Figure 2.2. Plot of number of hits/trial 
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Figure 2.3 Frequency plot of Number of Hits 
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Figure 2.4 Cumulative Success Ratio of Subject (both machines used) 
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The first plot (Figure 2.2) demonstrates the randomness of the number 
of hits while the second plot (Figure 2.3) demonstrates the frequency 
distribution takes on a "normal" appearance. The accumulative probability plots, at 
first glance,indicates that the subject was in a learning mode for the 
first five trials. A closer examination of the data fudicates chat this can 
occur naturally as part of the statistical distribution. 
The first three number of hits points are 7, 5, and 6 considering the 
first 75 points as the population with probability of success = .2936 
(the final probability) then the expected value is 22 (using binomial 
distribution) and the variance is 15.55 (S.D=3.9). As a normal deviation 
from the mean. (i.e., using normal distribution approximation P(x<18)=.13. 
Although the observed learning can be rationalized as a natural statistical 
deviation it warearteel further investigation. The plots of the accumulative 
probability of success for machine 1 and machine 2 are presented in Figure 2.5 
and Figure 2.6. The plot for machine 1 (Figure 2,5) is a typical sinesodial 
decreasing amplitude convergent curve. The plot for machine 2 however, is 
. very suspicious in terms of learning. The major peaks of the curve (at 
& nonponinetely trial 10, 23, 40 and 56) are increasing which implies his 
. aoopabelity-6r success is continuing to increase instead of converging on 
one point. Another interesting seine ieethat the points at which ” switches 
onto machine 2 are 1, 9, and 36. 
| Also of concern is the sharp upward turn during the last 8 samples. 
- The hits totals for this period, starting at sample 49 is 10, 10, 8 11, 6, 
8, 7, and 11 for a total of 71 hits out of a possible 200 far a probability 
of success of .36. Once again using the binomial distribution and using the 
probability of success of .29 (the cumulative probability up to the 49th 
seine the expected mean is 58 and the standard deviation 6.42. Using the 
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Figure 2.7 Plot of Number of Hits on Machine 1 
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normal approximation the SrobabiTity. P(X 71)=,02 of such an occurrence is 
quite low. 

| Although there are only 56 data points in this population and the 
apparent abnormalities are statistically possible (with low probability) this 
investigation concludes that the subject's learning for this case must be- 
flagged as a real possibility. Figure 2.7 (Number of hits on Machine 1) has 
been added to provide clarity. It appears that the subject just didn't have 
"low hit" days toward the end. | 


Learning within a Trial 


The question of learning within a trial or run fas been investigated 7 
Olga hott smaahinte 
by summing the number of hits of the Ith sample for the fun. The results 


cd 


are somewhat distorted because of the inequitable distribution of passes. 

The lower numbered samples have significantly more hits because of this. 2, ce 

A plot of the number of hits per sample vs. sample number is shown in Figure 2-77 
Notice that the first sample has a value of 34 hits. This means that 

everytime he ists down for a new 25 sample trial he hits 34% of the time on 


his first try. With this in mind along with the rest of the data points, it 


is obvious that the subject doesn't learn throughout the trial. 
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Figure 2.8 Total Number of Hits Within a Trial 
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Miscellaneous 

Numerous arrays of data have been examined for the purpose of obtaining 
some insight into the data. Some of the data is being printed herein so that 
the data can be examined more closely if desired. 


This first table is presented for use as a quick reference. 


Number 

Last of _ Machine 
Day Trial Tracks Used 
1 8 gg 8 
2 16 8 1 
3 24 8 2 
4 36 12 2 
5 44 8 2 
6 : 52 8 1 
7 56 4 1 
8 64 8 1 
9 68 4 1 
10 72 4 1 
1 76. 4 1 
12 80 4 1 
13 | 84 4 2 
14 88 4 2 
15 100 12 2 
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The following displays are presented below with little commentary. 


I. 


i. 


ITI. 


General trial summary (Figure 3.1). Each trial (25 choices) is 
listed with the following information, 

A. Machine used (1 or 2) 

B. Total number of machine states in each color (i.e., 6 yellow, 
6 green ....) for each trial. 

Total number of subject choices for each color for each trial. 
Total number of hits for each trial. 


Total number of passes for each trial. 


nm" moo 


Breakdown of hits by color. 

Machine data for machine 1 and machine 2 separately (Figures 3.2, 3.3) 
Just by examining these displays it may be possible to glean 4 
meaningful information. For example, machine 1 was used for the ae 
first 8 trials during which the first state of eacn trial was a 

yellow or red. If the first sample of each trial is most memorable, 
perhaps this is responsible for the subject's obvious preference of 

yellow and red (see Section 2 - Analysis of $2 Data Responses). 

Plots of the number of passes made. 

A. Number of passes vs. trial number (i.e., trial is 25 or more 

samples) (Figure 3.4) | 


_B. Number of passes vs. sample number (Figure 3.5) 


3] 
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Figure 3.2 Color states of machine 1 during the experiment 
(O-yellow, 1 areen. 2 blue. 3 red) 
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Figure 3.4 Total number of passes summed over a trial 
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Figure 3.5 Total number of passes summed over sample number 
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_C.. Number of passes and the number of hits vs. the trail number 
on one plot. Investigation of the hits/passes relationship 
was dropped when the coefficient of correlation between the 


two was computed at -.114 
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Figure 3.6 Plot of number of hits per trial and number of passes 


per trial 
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IV. Tables of state transitions which reflect the influence of the subject 


ww on the machine. For color choices of the subject the table shows the number 


of colors the machine has on the next sample. 


when the subject picked yellow, on the next sample 197 times the machine 


state was yellow. 


Yellow 
Green 
Blue 
Red 


Pass 


Yellow 
= Green 
Blue 
Red 


’ Pass 


Yellow 
Green 
Blue 
Red 


Pass 


Pee SUOMI asc namin recA Naik, AMAL AS I AAA MERON OH A OE Rt 


MACHINE STATES ON FOLLOWING SAMPLE 


Yellow 


88 
38 
27 
120 
84 


109 


58 
25 


“721 


146 


197 
. 96 


52 


241 


230 


Green 


77 
46 
28 
105 


83 


124 
47 
32 

125 

162 


201 
93 
60 

230 

245 


Blue 


87 


‘oe 


24 
99 


- 98 


128 
58 
a2 

136 

161 

215 
97 
66 

235 

259 
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For example on the first table, 


Machine 


Machine 


Both 
Machines 


Figure 3.7 State Transitions from Subject Choice to Future Machine State 
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V. Because of the possibility that the subject was learning the state of machine 
2 the distribution of the colors are plotted in Figures 3.8, 3.9, 4.0, and 
4.1. The only states used are those in which the subject didn't pass. 


Therefore there is a total of 25 for each trial. 
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Figure 3.8 Distribution of Yellow for Machine 2 
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Figure 4.0 Distribution of Biue for Machine 2 
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- Test Description Scoring 
: ae eee 
athe I A a et a a 
Halstead Category Nonverbal test requiring abstraction of conceptual relation- j 14 33 46 a 28 
Test , ships. Score: Total errors. 
Tactual Performance Requires placement of 10 geometrically shaped blocks in 
Test their correct locations on a formboard while blindfolded. 16.4 11.8 7.7 7.7 11.4 6.9 
Separate RT, LT, and bimanual trials. 
Score: Total time (min.). 
Speech Perception Discrimination of non-word speech sounds. 
4 2 0 2 5 3 
Test Score: Total errors. ; 
: et 
Seashore Rhythm Test Discrimination of nonverbal rhythms. Score: Number correct. | 27 25 28 29 26 29 


a a ee a ne 


Finger Tapping Test Measure of finger oscillation rate for 10-sec. period, both RT/LT RI/LT RI/LT RI/LT RI/LT- = RI/LT 
. RT and LT hand trials. Score: No. taps/10 sec. 53/50 53/49 48/47 54/53 47/47 48/43 


Trail Making Test Requires connecting numbered circles in order from 1 to 25. 


27 

(Part A) Paper and pencil task. Score: Total times (sec) #0 i aS ae 20 
Trail Making Test Requires connecting alphabetic and numbered circles by 56 50 55 50 54 53 

(Part B) alternating 1+A>+2>B, etc. Score: Total time (sec) 
Knox Cube Test Measure of attention span and immediate visual memory. 13 1A 13 16 17 V7 

Score: Number correct. 

Raven Progressive Nonverbal intelligence test involving spatial matrices. 39 53 49 55 60 54 

. Matrices Score: Number correct. 
eo ee 
Verbal Concept Requires abstraction of verbal conceptual relationships. 22 24 27 23 21 24 
Attainment Test Score: Number correct. 

Total: 


Buschke Memory Test Requires learning a 20-word list in a maximum of 12 trials with 
repetition of words omitted after each trial. Score: Max. no. 


words correctly remembered; List: no. words consistently remembered 


14/20 17/20 18/20 19/20 20/20 20/20 
List: 

8/20 14/20 11/20 16/20 15/20 16/20 

(8 trials) (7 trials 


“RT/LT RT/LT RT/LT RI/LT RT/LT RT/LT 
76/74 69/70 58/67 59/67 gad 48/50 


Grooved Pegboard Test] Requires insertion of 25 pegs in their holes in a pegboard. Both 


RT and LT hand trials. Score: Total time (sec). 


Spatial Relations 


Requires mental rotation and identification of figures 
Subtest of the PMA 


rotated in 2 dimensions. Score: no. correct - no. errors. 


60 52 


Gottschaldt Hidden Requires tracing outline of simple figure ee a 


Figures Test lines of more complexAppgaved Fer Release 200304H8.: Gabe BOP96- aan eee OU aor 


